The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.Comment: Best Student Paper Award, Proceedings of the 38th SGAI International Conference on Artificial Intelligence, Cambridge, UK, 2018, Artificial Intelligence XXXV, 201

fracture and microstructural study of bovine bone under mixed mode i ii loading

Abstract Understanding the fracture behavior and associated crack growth mechanism in bone material is an important issue for biomechanics and biomaterial researches. Fracture of bone often takes place due to complex loading conditions which result in combined tensile-shear (i.e. mixed mode) fracture mechanism. Several parameters such as loading type, applied loading direction relative to the bone axis, loading rate, age and etc., may affect the mixed mode fracture resistance and damage mechanism in such materials. In this research, a number of mixed mode I/II fracture experiments are conducted on bovine femur bone using a sub-sized test configuration called "compact beam bend (CBB)" specimen to investigate the fracture toughness of bone under different mode mixities. The specimen is rectangular beam containing a mid-edge crack that is loaded by a conventional three-point bend fixture. The results showed the dependency of bone fracture toughness on the state of mode mixity. The fracture surfaces of broken CBB specimens under different loading conditions were studied via scanning electron microscopy (SEM) observations. Fracture surface of all investigated cases (i.e. pure mode I, pure mode II and mixed mode I/II) exhibited smooth patterns demonstrating brittle fracture of bovine femur. The higher density of vascular channels and micro-cracks initiated in the weakened area surrounded by secondary osteons were found to be the main cause of the decreased bone resistance against crack growth and brittle fracture

Trace metals distribution and uptake in soil and rice grown on a 3-year vermicompost amended soil

This study was designed to investigate the influence of vermicompost (VC) on trace metals distribution and uptake in soil and rice plant in research field as split plot arrangement based on randomized complete block design with three replications in 2008. Main-plot was VC and chemical fertilizer (CF) that were added to soil in 6 levels (20 and 40 ton/ha VC, 20 and 40 ton/ha VC + 1/2 CF, CF and control). Application years considered as sub-plot comprised 1, 2 and 3 years. The results indicated thatfertilizers and application periods treatments influenced micronutrients in soil and rice. Available copper (Cu) had no significant difference under different treatments. The highest available iron (Fe) was found in the 40 ton treatment group. During the 3 years, application of 20 ton and enriched 40 ton gave the most available zinc (Zn) and manganese (Mn). In VC and enriched VC, treatments happened to give the highest Zn uptake by rice. Under the 3 years, application of 40 ton/ha VC, the highest Fe (91.19 ppm) and Cu (13.66 ppm) concentration was seen in flag leaf, while Fe (31.35 ppm) and Mn (27.56 ppm) was seen in grain. With the application of enriched 20 ton VC, the maximum uptake of Mn by flag leaf and Cu by grain was obtained

Life Cycle Assessment of Municipal Waste Management System (Case Study: Karaj, Iran)

LCA has been defined as a tool for evaluating the environmental burdens and potential impacts that can be applied to municipal solid waste management systems for determine the optimum municipal solid waste (MSW) management strategy.To investigate the Waste Management system strategyof Karaj City we used LCA method. Three scenarios were defined and compared based on environmental burden include water pollution, air pollution, consumed energy and waste residues.. For each of these scenarios, an ecological indicator was achieved from checklist values. From the environmental point of view, results show that recycling is one of the best alternatives for Waste Management. Furthermore, composting has an important role in alleviating the load of pollutants and energy usage in the Waste Management system. ©JASEMKeywords: Waste Management system, LCA, Kara

The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.The Dreaming Variational Autoencoder for Reinforcement Learning EnvironmentsacceptedVersionNivå

Novel linear piezo‐resistive auxetic meta‐sensors with low Young's modulus by a core–shell conceptual design and electromechanical modelling

Production of piezo-resistive auxetic sensors is usually carried out through mixing and coating methods. Although these methods are beneficial, Young's modulus of mixed sensors becomes high because of using a high percentage of sensing elements while the durability of coated sensors gets low due to the separation of sensing elements from the sensor surface. This article presents a new core–shell metamaterial model to address the mentioned problems. The shell and the core are produced of polydimethylsiloxane (PDMS) rubber and a mixture of PDMS/graphite powders (73.45 wt% graphite powders), respectively. A finite element model is developed via COMSOL software to predict the electromechanical behaviors of the created sensor and verified by an experimental study. Scanning electron microscope imaging is conducted to detect the separations of the graphite particles. The main important feature of this meta-sensor is to possess a linear sensitivity due to having zero Poisson's ratio. The advantage of this method is that Young's modulus of the sensor does not decrease (unlike the mixing method), and the sensor-coated particles do not separate from the sensor after a while (unlike the coating method). The introduced model has advantages that promote potential applications such as using sensory gloves to detect, for instance, human hand movements

Prediction of breast self-examination in a sample of Iranian women: an application of the Health Belief Model

<p>Abstract</p> <p>Background</p> <p>Iranian women, many of whom live in small cities, have limited access to mammography and clinical breast examinations. Thus, breast self examination (BSE) becomes an important and necessary approach to detecting this disease in its early stages in order to limit its resultant morbidity and mortality. This study examined constructs arising from the Health Belief Model as predictors of breast self examination behavior in a sample of women living in Bandar Abbas, Iran.</p> <p>Methods</p> <p>This study was conducted in eight health centers located in Bandar Abbas, Iran. The sample consisted of 240 eligible women who were selected from referrals to the centers. The inclusion criteria were as follows: aged 30 years and over; and able to read and write Farsi. Women with breast cancer, who were pregnant, or breast feeding, were excluded from the study. Data were collected by using a self administered questionnaire which included demographic characteristics and Champion's Health Belief Model Scale. This instrument measures the concepts of disease susceptibility (3 items), seriousness (6 items), benefits (4 items), barriers (8 items) and self-efficacy (10 items).</p> <p>Results</p> <p>The subjects' mean age was 37.2 (SD = 6.1) years. Just under a third of the subjects (31.7%) had performed BSE in the past and 7.1% of them performed it at least monthly. Perceived benefits and perceived self-efficacy of the women who performed BSE were significantly higher compared with women who did not practice BSE (p < 0.03). Furthermore, perceived barriers were lower among those who had performed BSE (p < 0.001). Logistic regression analysis indicated that women who perceived fewer barriers (OR: 0.70, 95% CI: 0.63-0.77, p < 0.001) and had higher self-efficacy (OR: 1.08, 95% CI: 1.02-1.13, p = 0.003) were more likely to perform BSE (R²= 0.52).</p> <p>Conclusion</p> <p>Findings from this study indicated that perceived barriers and perceived self-efficacy could be predictors of BSE behavior among the sample of women. Therefore, BSE training programs that emphasize self-efficacy and address perceived barriers are recommended.</p

Deep Reinforcement Learning: An Overview

In recent years, a specific machine learning method called deep learning has gained huge attraction, as it has obtained astonishing results in broad applications such as pattern recognition, speech recognition, computer vision, and natural language processing. Recent research has also been shown that deep learning techniques can be combined with reinforcement learning methods to learn useful representations for the problems with high dimensional raw data input. This chapter reviews the recent advances in deep reinforcement learning with a focus on the most used deep architectures such as autoencoders, convolutional neural networks and recurrent neural networks which have successfully been come together with the reinforcement learning framework.Comment: Proceedings of SAI Intelligent Systems Conference (IntelliSys) 201

LNCS

A controller is a device that interacts with a plant. At each time point,it reads the plant’s state and issues commands with the goal that the plant oper-ates optimally. Constructing optimal controllers is a fundamental and challengingproblem. Machine learning techniques have recently been successfully applied totrain controllers, yet they have limitations. Learned controllers are monolithic andhard to reason about. In particular, it is difficult to add features without retraining,to guarantee any level of performance, and to achieve acceptable performancewhen encountering untrained scenarios. These limitations can be addressed bydeploying quantitative run-timeshieldsthat serve as a proxy for the controller.At each time point, the shield reads the command issued by the controller andmay choose to alter it before passing it on to the plant. We show how optimalshields that interfere as little as possible while guaranteeing a desired level ofcontroller performance, can be generated systematically and automatically usingreactive synthesis. First, we abstract the plant by building a stochastic model.Second, we consider the learned controller to be a black box. Third, we mea-surecontroller performanceandshield interferenceby two quantitative run-timemeasures that are formally defined using weighted automata. Then, the problemof constructing a shield that guarantees maximal performance with minimal inter-ference is the problem of finding an optimal strategy in a stochastic2-player game“controller versus shield” played on the abstract state space of the plant with aquantitative objective obtained from combining the performance and interferencemeasures. We illustrate the effectiveness of our approach by automatically con-structing lightweight shields for learned traffic-light controllers in various roadnetworks. The shields we generate avoid liveness bugs, improve controller per-formance in untrained and changing traffic situations, and add features to learnedcontrollers, such as giving priority to emergency vehicles